Combining Sources of Information in Vision I. Computing Shape from Shading and Motion

Most of the basic problems in computer vision, as formulated, admit infinitely many solutions. But vision is full of redundancy and there are several sources of information that if combined can provide unique solutions for a problem. Furthermore , even if a problem as formulated has a unique solution, in most cases it is unstable. Combining information sources in this case leads to robust solutions. In this paper, we combine shading and motion to uniquely recover the light source direction and the shape of the object in view, and we describe how one could combine information sources present in the image in order to achieve uniqueness and robustness of low level visual computations. 0. Prolegomena Several problems in human or machine vision, as formulated , admit infinitely many solutions, because the available constraints that relate image properties to the unknown parameters are not enough to guarantee uniqueness. One approach to the solution of these problems is to regularixe them by requiring that the desired parameters satisfy some criteria (basically smoothness). Poggio et al. ([l], [2], [3]) have very successfully shown that several problems in early vision may be "solved" by standard regularization techniques. Another approach to the solution of ill-posed problems would be to look for more information sources in order to augment the number of physical constraints and achieve uniqueness of the parameters to be computed. For example, we can combine shading with retinal motion to compute the illuminant source and the shape of the object in view, combine shading with stereo, stereo with motion, texture with contour, and so on, in order to compute three-dimensional scene properties. The results of such an approach are summarized in Table 1 for reference. In the ellipses (top) are the different image cues (we take the liberty of calling stereo or motion a cue). By. a cue we mean a source of information, either coming from the image(s) or from the particular setup or condition of the visual system (stereo, motion). In the squares are the results that can be obtained (in terms of propositions) when information is combined from two or more cues. Two or more different cues are combined with arcs which lead to small circles containing pluses. Then, an arc leads from the "plus" to a square containing the result obtained from this combination. are highly appreciated. The interested reader is referred to [4]. IN this paper we study …